Robust characterizations of k-wise independence over product spaces and related testing results

A discrete distribution D over Σ1 × · · · × Σn is called (non-uniform) k-wise independent if for any subset of k indices {i1, . . . , ik} and for any z1 ∈ Σi1 , . . . , zk ∈ Σik , PrX∼D[Xi1 · · ·Xik = z1 · · · zk] = PrX∼D[Xi1 = z1] · · ·PrX∼D[Xik = zk]. We study the problem of testing (non-uniform) k-wise independent distributions over product spaces. For the uniform case we show an upper bound on the distance between a distribution D from k-wise independent distributions in terms of the sum of Fourier coefficients of D at vectors of weight at most k. Such a bound was previously known only when the underlying domain is {0, 1}. For the non-uniform case, we give a new characterization of distributions being k-wise independent and further show that such a characterization is robust based on our results for the uniform case. These results greatly generalize those of Alon et al. [STOC’07, pp. 496–505] on uniform k-wise independence over the Boolean cubes to non-uniform k-wise independence over product spaces. Our results yield natural testing algorithms for k-wise independence with time and sample complexity sublinear in terms of the support size of the distribution when k is a constant. The main technical tools employed include discrete Fourier transform and the theory of linear systems of congruences. ∗A preliminary version of this work, titled “Testing Non-uniform k-wise Independent Distributions over Product Spaces”, appeared in the Proceedings of ICALP 2010. †Research supported by NSF grants 0514771, 0728645,0732334, Marie-Curie International Reintegration Grant PIRG03-GA2008-231077 and Israel Science Foundation Grant Numbers 1147/09 and 1675/09. ‡Research supported in part by an Akamai Presidential Fellowship and NSF grants 0514771, 0728645 and 0732334.